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[57] ABSTRACT 

A projective display system includes a light source, a display 
surface, and a two dimensional light modulator array in an 
optical path between the light source and the display surface. 
TTie two dimensional light modulator array has a plurality of 
light modulators arranged in rows to modulate light from the 
light source, with al least one row being susceptible to 
failure. To compensate for any failed rows, a redundant light 
modulator array is provided, with modulated light output of 
the redundant light modulator array being directed by an 
optical element positioned to selectively substitute for at 
least one failed row of light modulatoi^. 

8 Claims, 7 Drawing Sheets 
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FIG. 3 
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FAULT TOLERANT PROJECTIVE DISPLAY 
HAVING REDUNDANT UGHT 
MODULATORS 

FIELD OF THE INVENTIGN 

The present invention relates to two dimensional image 
display systems. More particularly, the present invention 
relates to fault correction for non-woiking or damaged 
pixels of light modulator arrays using separately mounted 
redundant light modulator arrays. 

BACKGROUND AND SUMMARY OF THE 
INVENTION 

High resolution display systems based on discrete two 
dimensional arrays of reflective or transmissive elements 
(visually recognized as "pixels") are used in portable 
computers, LCD multimedia projectors, high definition 
televisions, flat panel displays, and virtual reality eyewear. 
As compared to conventional scanning or projection devices 
(e.g. cathode ray tubes or video projectors), such devices 
operate at low voltages, making them power efficient, and 
generally do not require long optical paths for operation, 
making them space efficient. 

However, widespread adoption of such devices for con- 
sumer display systems has been hindered by low manufac- 
turing yields. A typical two dimensional anay (using, for 
example, LCD's that control light transmission or micro- 
mirrors that control light reflection) may have a million or 
more discrete elements, with failure of only two or three 
elements making the display system useless. Even worse, 
initially operable two dimensional arrays may degrade in 
performance due to pinpoint screen damage, electronic 
failures, or mechanical wear and stiction problems. Particu- 
larly worrisome are display driver or mounting defects that 
can cause entire rows or columns of elements in the two 
dimensional array to simultaneously fail. 

What is needed is an apparatus and technique for provid- 
ing fault tolerance in two dimensional light modulator arrays 
of reflective or transmissive elements used in display sys- 
tems. Fault tolerance should allow for failure of single 
display elements, multiple display elements, or even failure 
of multiple rows or columns of display elements without 
appreciably diminishing display quality. Preferably, fault 
correction in such a fault tolerant system can be accom- 
plished dynamically, and failure of array elements may even 
be corrected in real time. 

The present invention provides such a fault tolerant 
system having a two dimensional light modulator array 
mounted on (or movable relative to) a fixed substrate. The 
light modulator array has a plurality of light modulators 
arranged in a first row and a second row, with the first row 
at a first row position with respect to the fixed substrate and 
the second row at a second row position with respect to the 
fixed substrate. A movable element attaches the two dimen- 
sional light modulator array to the fixed substrate, and can 
include, for example, piezovibralors, one or two dimen- 
sional mechanical steppers, rotary steppers, or other con- 
ventional mechanisms for mechanically moving the light 
modxilator array with respect to the fixed substrate. In 
operation, the movable element physicaUy moves the first 
row from its first row position to substantially occupy the 
second row position, effectively spreading light between 
operating elements of the light modulator array and failed 
elements. As will be appreciated, multiple rows can be 
displaced, with light being spread over three, four, or even 
more rows if desired. In conjunction with a light modulator 
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failure compensation system having an image reseating 
module to reduce apparent light modulator display intensity 
for selected display elements, and a light source intensity 

' control for increasing intensity of a variable intensity light 

5 beam, compensation for multiple clement failures, or even 
failures of entire rows, is possible. 

Physical displaconent of a Ught modulator array is not 
required for practice of the present invention. An alternative 
failure compensation mechanism using optical row displace* 
ment techniques includes a display system having a light 
source, a display surface, and a two dimensional light 
modulator array with a plurality of light modulators 
arranged in a first row and a second row to modulate light 
from the light source. Light modulated by the first row is 
presented as a first row of pixels on the display surface, and 
light modulated by the second row is presented as a second 
row of pixels on the display surface. One or more optical 
elements redirect first row modulated light from its presen- 
tation as the first row of pixels on the display surface to 
substantially occupy the second row of pixels on the display 

20 surface. In preferred embodiments this optical element can 
be a movable mirror in an optic path between the display 
surface and the two dimensional light modulator, however 
alternative mechanisms for optical row displacement can 
also be used. Such mechanisms may include, for example, 

25 use of two or more optical elements, each optical element 
having a light shutter, with a light path from each optical 
element to the display surface being alternately closable by 
each respective light shutter. In certain embodiments, the use 
of mechanical row displacement in conjunction with optical 

3Q row or column displacement is also contemplated. 

Two dimensional row and column displacement by vari- 
ous mechanical or optical mechanisms is also within the 
scope of the present invention. For example, light modula- 
tors arranged in a first column and a second column can be 

35 positioned substantially orthogonal to the first row and 
second row, with the first column at a first column position 
with respect to the fixed substrate and the second column at 
a second column position with respect to the fixed substrate. 
The movable element can then displace (mechanically or 

40 optically) the first column from its first column position to 
substantially occupy the second column position, as well as 
providing for row displacement for two dimensional move- 
ment. For example, the same virtual pixel can result from the 
periodic display of four physical pixels that are sequentially 

45 moved into position by displacement of the physical pixel 
array one pixel leftward, one row upward, one column 
rightward, and one row downward back to the original 
position. 

In operation, compensating for faults generally requires 

50 that each light modulator in a physical pixel array for 
displaying an image have substantially the same defined 
length to present a virtual pixel array with identical pixel 
sizes. The physical pixel aaay is moved by mechanical, 
optical, or opto-mechanical techniques that cause periodic 

55 displacement of the physical array of light modulators 
through at least the defined length of each of the light 
modulators with a periodicity generally exceeding about Mo 
of a second. Image processing software or firmware in the 
light modulator failure compensation system can correct for 

60 failure of one or more of the plurality of ligjbt modulators by 
shifting image position in coordination with periodic dis- 
placement of the light modulator array (physical anay), and 
reseating apparent light modulator display intensity for at 
least some light modulators positioned adjacent to a failed 

65 light modulator. 

As an example, consider a light modulator array (physical 
array) that periodically moves through a distance corre- 
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sponding to the length of a single light modulator. On 
average (ignoring travel time), each light modulator spends 
half of its time in an original position displaying a first 
virtual pixel, and the other half in a second, adjacent pixel 
displaying a second virtual pixel. Uncorrected, to a viewer $ 
the image would appear to blur as the pixel image (with 
associated gray and color value) initially in the first pixel 
was now di^layed in the first and second pixel. However, 
if the digital information sent to the physical anay was 
periodically diifted one pixel in coordination with the move- 
mcnt of the physical array, to a viewer the virtual pixel array 
would remain unblurred, even though two adjacent physical 
pixels are actually together creating the virtual pixel. Id 
other words, as each pixel was physically moved leftward 
one pixel, the pixel image being displayed was moved 35 
rightward by one pixel, causing each virtual pixel to appear 
to remain unchanged (and unblurred). 

Now, to correct for pixel failures the resealing modulator 
selectively varies image intensity of at least those pixels 
adjacent to, or in the vicinity of, a failed pixel. By appro- 20 
priate image display intensity adjustments (to all or some of 
the pixel elements in the physical pixel array), the image 
intensity can be varied over the whole (or part) of the light 
modulator array to maintain (or at least approximate) a 
correct relative image intensity for each virtual pixel in the 25 
array. In effect, image brightness of the light modulator array 
is reduced (in whole or in part) to eliminate or reduce point 
or line failure defects in light modulator arrays. If the light 
modulator array is used to reflect or transmit light from a 
variable light intensity source, compensation for loss of 30 
image brightness is possible by simply increasing brightness 
of the light intensity source. However, as will be appreciated 
by those skilled in the art, because of greater viewer accq)- 
tanoe of generalized or localized brightness variations as 
compared to point or line defects, tise of a variable intensity 35 
light source may not be required to satisfy viewer expecta- 
tions. Id such situations, selective image display intensity 
adjustments in the region of the failed pixels, with minimal 
or negligible corrections to far removed regions (for 
example, those regions positioned ten to a hundred pixels 40 
away from the failed pixel) may provide sufficient failure 
compensation for ordinary viewers of the display. 

In certain other embodiments of the present invention, 
fault correction can employ redundant rows or arrays that 
are optically or physically moved into position to fuUy or 45 
partially replace defective rows in a two dimensional Light 
modulator array. For example, a display system having a 
Hght source and a display surface (which can indude a 
reflective screen, transmissive screen, or even a human eye 
by use of direct retinal scarming) is used to display an image 50 
produced by a two dimensional light modulator array in an 
optical path between the light source and the display surface. 
The two dimensional light modulator array has a plurality of 
light modulators arranged in rows to modulate light from the 
light source, with at least one row having light modulators 55 
susceptible to failure. A redundant light modulator array 
having a second plurality of light modulators arranged in at 
least one row can be used to modulate Hght from the light 
source, with its light ordinarily being reflected away from 
the display surface, absorbed, or otherwise made oon- 60 
perceptible (for example, by shuttering to prevent light 
hitting the redundant light modulator array). However, if one 
or more light modulators fail, an optical element (such as a 
mirror, shutter, etc.), is operated to selectively direct light 
incident on the redundant light modulator array toward the 65 
display surface to substitute for at least one failed light 
modulator in the row of light modulators in the two dimen- 



4 

sional light modulator anay. Such a system advantageously 
allows for effectively replacing one Hght modulator, one or 
more rows of the two dimensional light modulator, or even 
large portions of the two dimensional light modulator with 
appropriately sized redundant hght modulators. As will be 
appreciated, redundant multiple rows edging (or 
surrounding) a two dimensional light modulator array can be 
moved by a movable element to replace edge positioned 
light modulators. In certain embodiments, it is contemplated 
to combine movable elements, redundant light modulator 
arrays, or even optical elements to provide various fault 
correction modes. 

Because of ease of manufacture, predictable performance, 
and durabihty, arrays of microelectromechanical pivoting or 
deformable mirrors controUably movable in response to 
applied electrostatic or electromagnetic forces are preferred 
for reflective displays. Such mirror systems are commer- 
cially available from Texas Instruments as part of a Digital 
Mirror Device (DMD) color projection system. 
Alternatively, switchable diffractive light valves such as 
developed by Silicon Light Machines, Sunnyvale, Calif, 
may be used. In addition, displays based on ferroelectric, 
electroluminescent, or twisted nematic Uquid crystals can 
also be used. Also suitable arc displays that can direcdy 
switch to an appropriate transmissive gray level without 
requiring use of time resolved gray levels such as needed for 
bistable reflective devices. Generally any device capable of 
modifying gray levels with a frequency exceeding about 10 
HZ, or more typically, with a frequency between about 100 
HZ and 1000 Hz, whether directly in response to variable 
analog current such as with liquid crystals, or through time 
based techniques such as employed by digital micromirrors 
or diffractive light valves, cem be used in the present 
invention. 

Additional functions, objects, advantages, and features of 
the present invention will become apparent from consider- 
ation of the following description and drawings of preferred 
embodiments. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a schematic representation of an exemplary two 
dimensional light modulator array attached to a fixed sub- 
strate by an intermediary movable element to allow for 
controlled physical displacement of the light modulators in 
row or column directions, with the light modulators being 
electrostatically actuated bistable mirrors or Uquid crystal 
light absorbers/transmitters that modulate light projected 
from a light source; 

FIG. 2 schematically illustrates a gray scale light modu- 
lator array (physical array) for modulating a variable inten- 
sity light source, with the light modulator displaying a 
virtual pixel array, with a single defective light modulator 
resulting in a dark display pixel; 

FIG. 3 schematically illustrates the light modulator array 
of FIG. 2 physically displaced by a single column to 
effectively share light between the dark display pixel and 
adjacent pixels; 

FIG. 4 schematically illustrates the light modulator array 
of FIG. 3 after image reseating to adjust image intensity of 
remaining pixels to match the shared grcyscale intensity of 
the dark display pixel and the adjacent pixel; 

FIG. 5 schematically illustrates the light modulator array 
of FIG. 4 after light intensity resealing to provide a desired 
greyscale intensity approximating the gray scale backgroimd 
of FIG. 2; 

FIG. 6 illustrates a light modulator array physicaUy dis- 
placed by multiple columns to effectively share light 
between the dark display pixel and several adjacent pixels; 
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FIG. 7 is a light modulator array physically displaced by ilhistrates reflectioa of an incident light ray 56 from light 

in two dimensions (sudi as the illustrated rectangular source 50 toward an absorber (not shown) using a mirror AS 

movement) to effectively share light between the dark pivoted through some angle. Reflection of an incident light 

display pixel and several row and column adjacent pixels; ray 52 from light source 50 toward a viewer (or projective 

HG. 8 is a flowchart schematically illustmUng modules 5 optics, not shown) is illustrated by light ray 54 reflected 

used for implementing a light modulator failure compensa- ^^J^ oppoalely ttlted miiror 46 A suitable posuion 

tion system' » r bistable minor (operated with 30 electrostatic actuators) for 

^-J . , .« . . . the present invention is described in U.S. Pat. No. 5^35,047 

HG. 9 Ilhistrates an opUcal oscillatory pixel displacement Hombcck, the disclosure of which is hereby expressly 

system; and incorporated by reference. As will be appreciated, altema- 

FIG. 10 illustrates an alternative fault tolerant light modu- tive reflective display systems can be employed, inchiding 

lator anay projection system having multiple redundant deformable mirrors, mirrors operated with electromagnetic 

rows or columns to substitute for non-functioning light actuators, or any other conventional reflective system Imown 

modidator rows or columns. to those skilled in the art. As with display 20, switching 

DETAILED DESCRIFHON OF TOE ^^Z"^^ ^^"^'^^f '^^'^^^8 ^ ^ 

i^E^iniE^^ L^^^^^^i wx controlled using control electronics 90. 

Switching speed for display elements of any of the 

FIG. 1 is a schematic, partially broken away view of a foregoing devices will generally be in the millisecond, 

fault tolerant display system 10 capable of supporting microsecond, or even nanosecond range, depending on the 

mechanisms for mechanically or optically compensating for 20 particular display architecnire. Advantageously, use of high 

failed, defective, or intermittently operating pixels in a speed switching techniques in conjunction with inherent 

display. For teaching purposes, FIG. 1 illustrates both a temporal smoothing of a viewer's eye (or other light inte- 

transmissive display system 20 and a reflective display 40 grative image sensor) allows for generation of gray levels 

supported by a movable element 14 on a common fixed using high speed switched bistable devices (stable reflective/ 

substrate 12. As shown, the transmissive display system 20 25 transmissive or non-reflectivc/absorptive), with the portion 

includes a row and colnmn array 24 of light transmissive of time that each display element is switched to reflect or 

display elements 28 to selectively transmit or absorb light transmit light being proportional to the perceived gray level, 

from a light source 22, while the reflective display system 40 For example, if the critical light flicker fusion frequency for 

includes a row and column array 42 of pivotal mirrors 44 a viewer is about 60 Hz, a display element switchable at 6 

that selectively alter reflective direction of light from light 35 KHz could switch about a 100 times between a light 

source 50. However, as will be appreciated, a given display reflective and a non-reflective state without appearing to 

win usually contain either transmissive elements (such as a flicker. By adjusting the proportion of time in each 

conventional active matrix LCD projector, LCD display, or reflective/non-reflective display state, the bistable display 

ferroelectric fluid display) or reflective elements (sudi as a element can be used to generate a gray level display. By use 

digital micromirror disclosed, for example, in U.S. Pat. No. 35 of colored light sources and such high speed switching it is 

5,535,047 to Hombeck), although certain embodiments may even possible to use generate full color displays such as 

combine transmissive and reflective elements for operation disclosed, for example, in U.S. Pat, No. 5,592,188 to 

(e.g. a stacked transmissive^flective display using LCD's Doherty et al., assigned to Texas Instruments, Inc. 

and a selectively deformable sheet mirror as discussed later Alternatively, systems of colored filters can be used to 

in connection with FIG. 10). 40 provide color, as well as gray level displays. 

In operation, each display element 28 in the light trans- Unfortunately, in practice it is diflBcult to manufacture 

missive display system 20 can be selectively controlled to large arrays having hundreds of thousands, or even millions, 

absorb incident light or allow its transmission to an image of the foregoing high switching speed display elements 

sensor such as a viewer's eye (either directly or in conjunc- without at least some individual display elements being 

tion with intervening projective optics, magnifiers, or 45 uncontroUable through mechanical or electrical failure. To 

reflectors, not shown). In FIG. 1, absorption of an incident compensate for such failures, one embodiment of the present 

light ray 32 by an absorbing display element 26 is shown, as invention includes a mechanism to physically displace rows 

is transmission of an incident light ray 30 through a trans- or columns in the display array in response to signals sent by 

missive display element 28. Each display element in the control electronics 90. As iUustrated in FIG. 1, this mecha- 

display system is designed to switch between gray scale 50 nism can include the movable element 14, a piezovibrator 

states in response to control signals sent by control elec- capable of independently moving in two orthogonal direc- 

tronics 90 (which can be patterned on the substrate through tions as shown by arrows 16 and 18. In alternative embodi- 

conventional electronic photolithographic techniques as ments the movable element can be a one dimensional 

flhistrated, or as a separate control module interfaced with piezovibrator, a movable linear stage, an angular arm, or any 

the display through appropriate control leads). As will be ss other conventional mechanism for periodically moving 

appreciated, various conventional light absorbers, ferroelec- transmissive system 20 (or system 40) through a distance, 

trie fluid displays, LCD displays, or other known light Movement is generally through a distance corresponding to 

display elements supporting at least two or more light integral multiples of each row or column distance in the 

transmissivity states (eg. 256 gray scale) can be used in the array 24 or anay 42. For example, in one embodiment, the 

present invention. 60 movable element 14 can physically move a certain distance 

Gray scale images can also be produced by controlling the sufficient to allow each row in the array 42 to occupy for a 

bistable position of each pivotal mirror 44 in the light brief time the position of an adjacent row. Alternatively, in 

reflective display system, with each mirror selectively con- addition to adjacent rows, the movable element 14 can be 

trolled to reflect incident light to an absorber (not shown) or operated to periodically displace a each row in the array 24 

allow its transmission to a viewer's eye (again, either 65 (or array 42) by two, three, or more rows with respect to its 

directly or in conjunction with inten^ening projective optics, initial starting position with respect to the fixed substrate 12. 

magnifiers, or reflectors, not shown). In FIG. 1, ray 58 As an extension to one dimensional movement, and as 
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iihistnted with respect to FIG. 1, the movable element cao 
displace an array 24 (or anay 42) in a columa direction, as 
well as in a row direction, to give full two dimensional 
movement 

Cbmpensation for failure of individual or multiple display 
elements in such a fault tolerant display 10 is best described 
in conjunction with FIGS. 2, 3, 4, and 5, which shows 
compensation for a single failed display element 124 in a 
sdiematically illustrated display 110. As will be appreciated, 
although the following description of operation of the 
present invention is based on a portion of a single row, pixel 
displacement can be arbitrary, with row displacement, col- 
umn displacement, angular displacement, displacement in 
pixel blocks (using, for example, combinations of row and 
column displacement) being used. All functioning display 
elements are shown in a light transmissive (or light 
reflective) state, with the failed display element 124 being in 
a light absorptive (or light non-reflective) state. In addition, 
it will be appreciated that the present invention is not limited 
to substantially square or rectangular display pixels, with 
circular, elliptical, hexagonal, or other suitable geometries 
constituting contemplated embodiments of the present 
invention. Because of wide availability however, typically 
an array of rectangular display elements arranged in a 
rectangular grid are used (eg. 1024 rows by 768 columns of 
rectangular display elements). 

For ease of Ohistration, in FIG. 2 only a portion of a single 
row of display 110 in the vicinity of the failed display 
element 124 is shown. The display 110 can be transmissive, 
reflective, emissive, a combination of transmissive and 
reflective, or any other display architecture able to provide 
high speed gray scale adjustment to individual pixels. The 
display 110 will typically include a two dimensional array of 
display elements for transmissive or reflective display of an 
image. To aid in understanding the present invention, FIGS. 
2 through 5 show a variable light intensity source 114 and a 
one dimensional partial section of a two dimensional physi- 
cal pixel array 120 consisting of an array of display elements 
121-127 (including failed display element 124). As wiU be 
appreciated, even though the following discussion is made 
in terms of gray levels, by use of fixed or movable colored 
filters, colored emissive displays, or other suitable color 
producing mechanism, correction of faulty colored display 
elements is within the scope of the present invention. 

Display elements 121-126 are each controlled to display 
a white pixel in the absence of unintended display element 
failures, while display element 127 is controlled to provide 
a gray pixel (e.g. by time modulation of bistable reflective or 
transmissive elements, or direct gray scale presentation with 
LCD's). The display elements 121-127 collectively provide 
a virtual pixel array 130 consisting of the pixel image of 
display elements 131-137 developed by application of the 
variable intensity light source 114 to the physical pixel array 
120. The intensity of the light source is indicated within the 
variable light intensity source 114 (intensity 100 in FIG. 2, 
in arbitrary units), the percentage transmissivity/reflectivity 
of each display element 121-127 is respectively indicated 
(eg. 0%, 50%, 100%), and the perceived gray level of the 
virtual pixel array 130 shown (on a scale of 0 to 100, with 
100 being the brightest and 0 being black). In eflect, the 
virtual pixel array 130 is what a viewer would see when 
looking at the display, even if the physical pixel array 120 
is undergoing high speed movement in conjunction with 
image processing driven pixel intensity remapping as dis- 
cussed in conjunction with FIGS. 3-5. 

As can be seen by inspection of FIG. 2, display element 
124 of the physical pixel array 120 has failed, so that the 
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virtual pixel array 130 has a corresponding dark pixel 
element 134. However, if the failure of display element 124 
can be dctermiaed, remedial measures to mitigate or entirely 
correct the failure can be employed. As seen in FIG. 3, the 
light modidator array of FIG. 2 can be physically displaced 
by a single column to efifectively share light (through time 
averaging) between the daik display pixel 134 of the virtual 
pixel array 130 and an adjacent pixel 135. The physical 
displacement is both high speed and oscillatory, with the 

] physical pixels being displaced one column and returned to 
their original position several tens, hundreds, thousands, or 
even tens of thousands times each second. In a typical 
embodiment of the invention, oscillations on the order of 30 
to 100 Hz are generally suitable. Absent any image 

5 processing, the virtual pixel array 130 now does not have a 
single dark pixel 134, but instead blurs the anay to have two 
gray pixels 134 and 135, each virtual pixel having a gray 
level of 50 instead of Uie desired gray level of 100. In 
addition, the originally gray pixel 137 (having an original 

0 gray level of 20 derived from pixel 127 as seen in FIG. 2) 
derives half of its share of light with the adjacent physical 
pixel element 126, brightening the virtual display pixel 
element 137 from a gray level of 20 to a gray level of 60 
(20%+ 100% divided by 2 to equal 60% Ught transmission/ 

5 reflection/transmission). 

Providing a deblurred, rescaled image with the proper 
relative gray scale for a virtual pixel di^lay elements is 
schematically illustrated in FIG. 4. As seen in FIG. 4, image 
grayscale of pixels 121, 122, 126, and 127 are scaled down 

0 to 50 to match the shared grayscale intensity of the dark 
display pixel and the adjacent pixel lb prevent unwanted 
brightening of virtual pixel element 133, pixel 123 is scaled 
to have a grayscale intensity of 50 while physically in a 
position that provides virtual pixel 133, and a gray scale 

5 intensity of 100 when moved one column leftward into the 
position that generates virtual pixel 134. Similarly, pixel 125 
is scaled to have a grayscale intensity of 100 while physi- 
cally in a position that provides virtual pixel 135, and a gray 
scale intensity of 50 when moved one column leftward into 

0 the position that generates virtual pixel 136. As a result, the 
virtual pixel array 130 now displays a uniform grayscale 
intensity of 50 for each pixel element 131-136. In addition, 
"to provide a correct gray scale for virtual pixel 137, pixel 
126 must be scaled to have a grayscale intensity of 50 while 

5 physically in a position that provides virtual pixel 136, and 
a gray scale intensity of 10 when moved one column 
leftward into the position that generates virtual pixel 137, As 
seen in FIG. 5, this rescaled image (with elements 131-136 
having a perceived grayscale of 50, and element 137 having 

0 a perceived gray scale of 10) is corrected for brightness by 
doubling (100 to 200) the hght intensity of the light intensity 
source 114. This brings the perceived gray level back to the 
target value of 100 for pixels 131-136 (and 20 for pixel 137) 
in the virtual pixel array 130. 

5 As will be appreciated, oscillatory movement of a physi- 
cal pixel array 120 is not limited to single pixel displace- 
ments. FIG. 6, which substantially corresponds to the 
embodiment of the invention illustrated in FIG. 3, shows 
displacement of the physical pixel array 120 by three pixel 

0 elements. In contrast to the embodiment illustrated in FIGS. 
2-5, the target value of aU pixel elements in FIG. 6 is 100, 
with the single failed pixel having a gray level of 0. The 
shown displacement allows light firom a single failed modu- 
lator to be spread among three pixels, decreasing the 

5 required intensification of the light intensity source 114 to 
bring the perceived level back to its target gray level value 
of 100. In certain embodiments, oscillatory pixel displace- 



06/28/2004, EAST Version: 1.4, 1 



5,8: 

9 

ment can coirespond to five, ten, or more cohmms (or rows) 
using suitable actuators and pixel arrays having a suitably 
high switching speed. 

In yet another embodiment of the invention, osdllatory 
movement can be two dimoisional. As seen in FIG. 7, which s 
substantially corresponds to the embodiment of the invea- 
tion illustrated in FIG. 3, shows di^lacement of the physical 
pixel array 120 in two dimensions. Movement is by sequen- 
tial and repeated column displacement of one pixel leftward, 
one row upward, one column cightward, and one row 
downward back to the original position. Again, the target 
value of all pixel elements is 100, with the single failed pixel 
having a gray level of 0. This two dimensional displacement 
allows light from a single failed modulator to be spread 
among four adjacent pixels, decreasing the required inten- 
sification of the light intensity source 114 to bring the 
perceived level back to its target gray level value of 100. In 
certain embodiments, oscillatory pixel displacement can be 
varied to have other oscillatory patterns. For example, 
movement could be by sequential and repeated column 
displacement of ten pixels leftward, one row upward, ten 
columns rightward, and one row downward back to the 
original position, to provide a high aspect rectangular pixel 
displacement orbit (as compared to the square orbit of HG. 
7). Other repeating patterns can of course be employed as 
desired. 

To aid in understanding the present invention, the fore- 
going image processing and display mechanisms are sche- 
matically illustrated with reference to FIG. 8. In operation, 
a light modulator failure compensation system 80 requires a 
light modulator failure identification unit 81 to identify 
malfunctioning light modulators (physical pixels in a one or 
two dimensional physical pixel array). Failure identification 
can be a one-time electronic or optical inspection at a 
manufacturing facility, or more preferably can be made "oo 
the fly" during device warm-up or operation. Optical iden- 
tification can be as simple as inspecting an array to see 
which element is anomalously bright (or dark). Electronic 
identification can rely on addressing each light modulator to 
switch optical states (gray levels) while simultaneously 
measuring capacitance, resistance, or any other suitable 
electrical switching response properties, with failures being 
identified by inappropriate electrical responses. For 
example, an electrostatically actuated micromirror held in a 
non-reflective state by stiction, damaged supports, or block- 
ing debris would not report a normally expected capacitance 
change during switching, allowing it to be identified as a 
failed light modulator. 

If any light modulators have failed, the oscillatory pixel 
displacement unit 82 is utilized to distribute fight from two 
or more physical pixels to each respective virtual pixel in a 
virtual pixel array seen by a viewer. The physical pixel array 
is moved by mechanical (as seen in FIGS. 4-7), optical (as 
seen in the following FIG. 9), or opto-mechanical techniques 
(as seen in the following FIG. 10) that cause periodic 
displacement of the physical array of light. The extent, 
directionality, speed, and path of physical pixel array move- 
ment can be controlled by on-board or external digital or 
analog controllers, and can be optionally adjusted to provide 
for better image compensation. For example, a one dimen- 
sional movement of the physical pixel array with a two or 
three pixel movement shift may be initially adequate for 
light distribution. However, as additional light modulators 
fail, two dimensional movement at a higher speed may be 
required to provide adequate image quality. 

Positional and velocity information from the light modu- 
lator failure identification unit 81 and the oscillatory pixel 
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displacement imit 82 is passed to image processing software 
or firmware in the image shift compensator 83. Again, the 
software can be onboard (directly formed on the substrate as 
seen in FIG. 1), or can be a separate memory and logic 
module (digital or analog) connected to the oscillatory pixel 
displacement unit and the electronics necessary for driving 
pixel image display. The compensator 83 is used to shift 
image position in coordination with periodic di^lacement 
of the oscillatory pixel displacement unit 82. In operation, 
the digital information used to provide a pixel image is 

^ periodically shifted one or more pixels in coordination with 
the movement of the physical array, so that two or more 
physical pixels together create a virtual pixel. For example, 
as eadi pixel was physically moved leftward two pixels, the 
pixel image display was moved rightward by two pixels, 

5 causing each virtual pixel to appear to remain unchanged 
(and unbhirred), even though three physical pixels are 
actually providing fight to a single virtual pixel. 

As will be appreciated by those skilled in the art, the 
image shift compensator 83 can optionally provide for 

0 subpixel display precision using various hyperacuity tech- 
niques. For example, sub-pixel edge placement is enabled if 
specific light modulators are turned on not only while at 
integral pixel distance intervals, but for some amount of time 
while at a fractional distance from each integral pixel 

5 position. While generally requiring a higher frequency of 
oscillation of the physical array, hyperacuity techniques 
based on timing can arbitrarily increase effective display 
resolution, and arc particularly useful for minimizing abas- 
ing features that are easily detected by human viewers. 

Q Information from units 81, 82, and 83, including any 
optional hyperacuity timing information, is passed to an 
image intensity rcsealing unit 84. As previously described in 
connection with FIGS. 2-5, the resealing unit 84 modifies 
some or all of the displayed gray levels of light modulators 
in the physical pixel array. The image intensity resealing imit 

^ 84 controls this gray level adjustment, with those Ught 
modulators positioned adjacent to a failed light modxilator 
generally having the greatest differential gray level 
adjustment, and/or requiring rapid switching of gray levels 
(depending on their physical pixel position relative to the 

0 virtual pixel). 

The loss in overall brightness can optionally be corrected 
by use of a variable light source intensification imit 85 
(corresponding to fight source 50 of FIG. 1). Compensation 
for loss of image brightness is possible by simply increasing 

5 brightness of the Hght source intensification unit 85. Such 
brighmess correction is easier if a large number of physical 
pixels provide each virtual pixel, with a 2:1 physical to 
virtual pixel ratio (such as discussed in connection with 
FIGS. 2-5) requiring a doubling of fight source brightness, 

0 while a 10:1 ratio only requires about a 10% increase in 
brightness to provide a fully corrected image 86. As wiU be 
appreciated by those skilled in the art, because of greater 
viewer acceptance of geneiafized or localized brightness 
variations as compared to point or line defects, use of a 

5 variable light source intensification unit 85 may not be 
required to satisfy viewer expectations. In such a case, an 
approxunately corrected image 86 will be di^layed, with 
brightness losses in the vicinity of failed pixel elements 
being imperceptible or at least not bothersome to a viewer. 

iO In addition, the present invention contemplates selective 
image display intensity adjustments in the region of the 
failed pixels, with minimal or negligible corrections to far 
removed regions (for example, those regions positioned ten 
to a hundred pixels away from the failed pixel). Again, 

15 because of known fimits to human visual perception, such 
Umited compensation may provide adequate fault correction 
for viewers. 
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As will be ^predated, movement of the two dimensional 
light modulator array (whether mechanical or optical) does 
not have to be exchisivcly through stepwise translation. 
Instead, movement may occur continuously over a distance 
of multiple pixels, with acceleration and deceleration occur- 
ing when a change of direction of motion is required. Such 
a continuous motion profile would result in the physical 
pixels occupying specific virtual pixel tocations for different 
amounts of time. With appropriate modifications, the image 
shift compensator unit ^ can utilize information about the 
position and velocity of the effective anay motion to deter- 
mine the appropriate image shift and intensity corrections 
such as described in connection with FIG. 8. For example, 
an implementation which utilizes oscillatory translation over 
a distance of 20 pixels may have uniform velocity in the 
center of the oscillation, but will move slower at the end- 
points. The image shift compensator can obtain information 
about the motion through position feedback, through veloc- 
ity feedback, or through other means of prescribing motion, 
such as a constraining motion to be a function of time, and 
make corrections as necessary to display a deblurred image. 

In addition to physically moving the array (such as 
discussed in connection with FIG. 1) to create a correspon- 
dence of two or more physical pixels to each virtual pixel, 
it is possible to create such a correspondence in conjunction 
with various optical or optomechanical techniques. For 
example, FIG. 9 illustrates a schematic, partially broken 
away view of a fault tolerant display system 210 capable of 
supporting mechanisms for mechanically or optically com- 
pensating for failed, defective, or intermittently operating 
pixels in a display. For teaching purposes, FIG. 9 illustrates 
a mirror based reflective display 240 having row and column 
physical anays 242 of pivotal mirrors 244 fixedly mounted 
to rotate with respect to a substrate 212. However, as those 
skilled in the art will appreciate, various other reflective or 
transmissive display devices, including those discussed in 
connection with FIG. 1, can be used in the present invention. 

In operation, a virtual gray scale pixel array image 280 on 
a screen 282 can be produced by controlling the bistable 
position of each pivotal mirror 244 in the physical array 242, 
with each mirror selectively controlled to reflect incident 
light to an absorber (not shown) or allow its transmission to 
screen 282 via a pixel image displacement mirror 270 
(pivotably mounted on an actuator 272) either directly or in 
conjunction with intervening projective optics, magnifiers, 
or fixed reflectors (not shown). As shown in FIG. 9, an 
incident light beam 256 fi-om light source 250 strikes the 
physical array 242 of Ught modulators, and is reflected in a 
desired image pattern toward mirror 270. The mirror 270 can 
be routed (for example, at a 10 Hz, 100 HZ, 1000 Hz, or 
greater frequency) by an actuator 272 to displace the beam 
by a distance corresponding to one or more virtual pixels of 
the virtual gray scale pixel array image 280. Operation of 
actuator 272 for pivoting the mirror in direction 274 can be 
enabled in either one, two, or three dimensions by electro- 
static attraction, electromagnetic attraction, a piezovibrator, 
a movable linear stage, an angular arm, or any other con- 
ventional mechanisin for periodically effecting rotation or 
movement of mirror 270. 

In a manner similar to that previously discussed in con- 
nection with FIG. 1, on-board control electronics 290 can be 
used for implementing various image processing module 
units previously described in connection with FIG. 8, includ- 
ing mirror fault identification, image resealing, optional 
hyperacuity processing and control of mirror 270 for virtual 
pixel movement. Movement of mirror 270 is generally 
through a distance corresponding to integral multiples of 
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each row or column distance in the virtual gray scale pixel 
array image 280. For example, in the illustrated 
embodiment, the mirror 270 can rotate a certain distance 
sufficient to allow each row 284 in the array 280 to occupy 

5 for a brief time the position of an adjacent row 286. As will 
be appreciated, by use of a two dimensional movement of 
mirror 270, full two dimensional movement pixel displace- 
ment is also possible. 
RG. 10 illustrates yet another embodiment of the present 

]0 invention for fault tolerant correction of damaged physical 
pixel arrays. Fault tolerant display system 310 supports 
redundant arrays 385 mounted on substrate 312 for mediani- 
cally or optically compensating for failed, defective, or 
intermittently operating pixels in a display. System 310 

J 5 includes a mirror based reflective display 340 having a row 
and column physical array 342 of pivotal mirrors 344 
mounted for bistable rotation with respect to a substrate 312. 
The reflective display 340 is further mounted on a movable 
element 314, similar to that discussed in connection with 

20 FIG. 1, for supporting oscillatory movement of array 342. As 
those skilled in the art will appreciate, various other reflec- 
tive or transmissive display devices, including those dis- 
cussed in comiection with RG. 1, can be used in the present 
invention. In addition, multiple redundant rows or columns 

25 are contemplated for operation of alternative embodiments 
of the present invention, with complete duplication of all 
rows or columns of physical array 342 being within the 
scope of the invention. 
In operation, a virtual gray scale pixel array image 380 on 

30 a screen 382 can be produced by controlhng the bistable 
position of each pivotal mirror 344 in the physical array 342 
of light modulators, with each minor selectively controlled 
to reflect incident light to an absorber (not shown) or allow 
its transmission to screen 382 to provide a virtual pixel array 

35 image 380. In the absence of any pixel defects, incident light 
beam 356 from light source 350 striking the redundant array 
385 of light modulators is ordinarily reflected away from the 
screen 382. If there are a few point light modulator defects, 
the physical array 342 can be oscillated (by piezovibrator 

40 314 in directions 316 and 318, for example) and subjected 
to various image processing techniques such as discussed in 
connection with FIG. 8 to provide adequate fault correction. 
However, if there are a great many adjacent light modulator 
failures in the physical array 342 resulting in non-operative 

45 virtual pixels (for example, those failed virtual rows 386), 
the foregoing described techniques may be unable to fully 
compensate for the faults. 

Fortunately, the embodiment of the invention described in 
relation to FIG. 10 allows for large scale compensation of 

50 numerous row or column faults. A mirror 370 can be rotated 
in direction 374 by actuator 372 to map the reflected beam 
from the redundant anay 385 from its initial virtual position 
(rows 384) onto the position of failed virtual rows 386. like 
the embodiment of the invention discussed with reference to 

ss FIG. 9, operation of actuator 372 for pivoting the minor can 
be enabled in either one, two, or three dimensions by 
electrostatic attraction, electromagnetic attraction, a 
piezovibrator, a movable linear stage^ an angular arm, or any 
other conventional mechanism for periodically effecting 

60 rotation or movement of mirror 370. The mirror 370 can be 
fixed in position to direct a virtual pixel image from redun- 
dant rows to overlap rows of failed pixels. This can be a one 
time adjustment of mirror position, or more typically the 
mirror can be dynamically moved after operation of light 

65 modulator failure identification unit (such as described in 
connection with FIG. 8) to create an overlap of the redun- 
dant virtual pixels and the failed virtual pixeK In certain 
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embodimeats of the inveation, large numbers of redundant 
rows, including as many redundant rows as are present in the 
physical array 342, can be positioned beside the physical 
array. In practice, two highly defective physical arrays can 
be positioned to project an overlapping virtual array, with 
the second physical array turning on (when required) only 
those pixels that are positioned to replace failed pixels in the 
first physical array. Such an embodiment of the present 
invention allows two costly, yet defective, physical arrays to 
cooperate to emulate a non-defective array. Since failure 
rates currently approach 80% to 90% for many varieties of 
high pixel density physical arrays, the present invention of 
combining two defective physical arrays can be cost effec- 
tive. 

As seen in FIG. 10, overlapping virtual projections from 
redundant rows and the physical array can be employed in 
conjunction with an oscillatory fault compensation mecha- 
nism sudb as describe in connection with FIGS. 1-8. In those 
situations where the physical array 342 of light modulators 
is oscillating in directions 316 or 318 (as a result of 
operation of piezovibrator 314), the mirror 370 can be 
rotated in one or two dimensions to track oscillatory move- 
ment of the physical array, in a manner similar to that 
described in connection with FIG. 9. This allows large scale 
row defects to be completely compensated for by virtual 
projections of redundant arrays, while isolated pixel failures 
can be corrected for using oscillatory movement and image 
processing such as discussed in conoection with FIGS. 1-8. 
In addition, as those skilled in the art wUl appreciate, 
oscillatory movement allows for slraighforward replacement 
of failed pixels on an array edge. For example, the physical 
array can be surrounded by ten or twenty rows (or columns) 
of redundant arrays that are transiently moved into position 
to provide replacement virtual pixels. 

As those skilled in the art will appreciate, other various 
modifications, extensions, and changes to the foregoing 
disdosed embodiments of the present invention are contem- 
plated to be within the scope and ^irit of the invention as 
defined in the following claims. 

What is claimed is: 

1. A display system comprising 

a light source, 

a display surface, 

a two dimensional light modulator array in an optical path 
between the light soin-ce and the display surface, the 
two dimensional light modulator array having a plu- 
rality of light modulators arranged in rows to modulate 
light from the light source, with at least one row being 
susceptible to faihire, 
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a redundant light modulator array having a second phi- 
raiity of light modulators arranged in rows to modulate 
light from the light source, and 

an optical element positioned to selectively direct light 
5 incident on the redundant light modulator array toward 
the display surface to substitute for at least one failed 
row of light modulators in the two dimensional light 
modulator array. 

2. The display system of claim 1, wherein the plurality of 
10 li^t modulators reflect light. 

3. The display system of claim 1, wherein the optical 
element selectively directs light incident on at least two rows 
of the redundant light modulator array to substitute for at 
least two failed rows of light modidators in the two dimen- 

15 sional light modulator array. 

4. The display system of claim 1, further comprising an 
actuator to move the optical element between predetermined 
positions. 

5. The display system of claim 4, wherein the two 
20 dimensional light modulator array is movable with respect to 

the optical element, and wherein the optical element is 
movable to track movement of the two dimensional light 
modulator anay. 

6. The display system of claim 5, wherein the actuator 
25 moves the optical element at a frequency greater than about 

10 kHz. 

7. The display system of claim 1, wherein the redundant 
light modulator has substantially as many light modulators 
as the two dimensional light modulator array. 

^0 g. A display system comprising 
a substrate, 

a two dimensional light modulator array movably 
mounted with respect to the substrate, the two dimen- 
2^ sional light modulator having a plurality of light modu- 
lators arranged to form at least one row to modulate 
light, 

a redundant light modulator array movably mounted with 
respect to the substrate, the redundant light modulator 
40 having a plurality of light modulators arranged to form 
at least one redundant row to modulate light, with the 
redundant row positioned adjacent to the at least one 
row of the two dimensional light mod\ilalor array, and 

a movable element for moving the two dimensional light 
45 modulator array and the redundant array relative to the 
substrate, with the movable element moving the redun- 
dant row to transiently occupy the position of at least 
one row of the two dimensional light modulator array. 

« * * * « 
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